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Abstract 

The purpose of this study was to investigate the achievement and attitudinal effects of learner patterns of navigational 
behaviors and option choices in using a hypermedia-based computer-assisted instruction (CAI) unit designed for college 
students. Also investigated were the effects of two incentive structures on achievement and attitudes, as well as the 
relationships among prior interest, experience, achievement and attitudes. Students were randomly assigned to either a 
task-incentive or performance-incentive version of a free-access, HyperCard format instructional computer program. 

Multiple regression analysis revealed that total time spent on instruction and practice and the number of times previous 
and back selections were made were significant predictors of achievement. Type of incentive did not yield a significant 
difference in posttest scores or attitudes between treatments. Overall, subjects had positive attitudes toward the 
instructional program. A positive relationship was found between prior experience with CAI and achievement. The 
results of this study suggest that college students may be able to accurately judge in what fashion they need to review 
previously viewed screens in computer programs in order to receive higher posttest scores. 

Introduction 

Learner control in computer-assisted instruction refers to the control of options within programs (Hicken, 
Sullivan, & Klein, 1992). Despite the attractiveness and cognitive appeal of learner-centered instruction, results of 
research'on learner control to date have been inconclusive. For example, when given control over content review (Kinzie, 
Sullivan, & Berdel, 1988), pacing (Gray, 1987), sequencing (Steinberg, 1989), and presentation medium (Ross, 

Morrison, & Odell, 1989) students gained higher achievement scores. However many studies have indicated no 
significant differences between learner control or program control (Klein & Keller, 1989) or poorer learning when 
students control their own learning (Fry, 1972). As a result, Hannafin (1984) and others have called for caution in giving 
students control over CAI programs. 

Based on the work of Carrier and her colleagues (cf. Carrier, Davidson, & Williams, 1985; Carrier, Davidson, 
Williams & Kalweit, 1986) virtually all of the more recent learner-control research has been conducted by providing 
students with programs in which they either add or by pass instructional elements. Researchers have begun to use 
programs, called Full-Minus or Lean-Plus, in which students can add or bypass e.^amples, practice items and review 
(Hicken, Sullivan, & Klein, 1990; Igoe, 1993; Hannafin, 1993). Results of these studies have been inconclusive with 
regard to the effect of learner control on achievement. 

More consistently favorable results have been obtained in studies correlating student attitudes and learner control. 
Pascal (1971) found that students who could choose the format in which instruction was presented enjoyed the subject 
matter more. Fisher, Blackwell, Garcia and Greene (1975) reported that students who could choose the difficulty and 
amount of math practice became more engaged in the instruction than those who could not. Morrison, Ross, and 
Baldwin (1992) found that learners allowed to choose the amount and context of practice problems had more positive 
attitudes than learners in a program-controlled treatment. 

A factor that has not been widely studied in learner-control studies is incentive to perform. Tennyson and 
Buttrey (1980) observed that the use of an actual classroom-related incentive, that is, a grade, was missing from most 
learner-control studies. Whereas a grade would have provided a performance-contingent incentive, many studies have 
utilized only a task-contingent incentive, in which simple completion of the task constituted student performance. In 
comparing task-contingent incentives and performance-contingent incentives, Hicken et al. (1992) found that the 
performance-contingent incentive group performed better on the posttest. When comparing types of performance- 
contingent incentives, Igoe (1993) found that there were no significant differences between students who selected the grade 
they wanted to achieve and those who were assigned a specific achievement level. Hicken et al. (1992) advised that 
further research on incentives in learner-control studies should be conducted. 

Many of the instructional programs created for learner-control studies have used hypermedia technologies such as 
HyperCard and Toolbook. In general, these programs have been primarily linear in format (Freitag & Sullivan, 1995; 
Schnackenberg, Sullivan, Leader, & Jones, 1995). They have not allowed learners to go back at any time to previous 
instruction or to branch to other material within the program. Little research has been conducted in which the learners 
have been allowed to utilize the network-structure of hypermedia. This network-structure frees learners to explore 
instructional programs using any options and navigational patterns they desire. 

Several studies investigating the use of navigational aids in non-linear programs have recently utilized this 
network structure. When comparing a navigation system that involved jumping to and from an index separate from text 
displays to one that presented navigation options directly on the text pages, Wright and Likorish (1990) found that adult 
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subjects preferred index navigation in a book-like hypertext program and page navigation for information that was in the 
form of a matrix. Stanton and Stammers (1990) reported that adults who were trained using a non-linear program were 
able to carry out a simulated task significantly better than those who were trained using a linear program. Additionally 
when comparing the cffecti\eness of using analytical i^proaches or browsing techniques to find information in a 
hypertext environment, Maohionini and Shneiderman (1988) found no significant differences among high-school age 
students. 

The purpose of the present study was to investigate the achievement and attitudinal effects of learner patterns of 
navigational ^haviors and option selections while they used a hypermedia-based instruction unit designed for college 
students. Also investigated were the effects of varying incentive structures on achievement and attitudes, as well the 
relationships among prior interest, experience, learner achievement and attitudes. Finally, qualitative data regarding 
students* perceptions of how they used the program and other attitudinal data were collected. 

y Method 



Subjects 

Participants in the study were 1 10 undergraduate students, mostly education majors, enrolled in an introductory 
computer course at a large southwestern university. The study took place iate in the semester, therefore students had 
already used IBM-PC computers to learn about DOS, word processing, databases, and spreadsheets. Students were told 
that this section of the course would introduce them to using Macintosh computers and to the concept of “CAI**, that is 
computer-assisted instruction for education. The instructional program constituted a regular assignment for the course, 
for which they would earn points toward their grade. 

Materials 

The instructional program was modified from one created by Robert Lievens. It was designed to teach various 
facts about the prehistoric Anasazi people of the American southwest in a computer-assisted, Macintosh-based HyperCard 
format. A computer-based data collection program ( twyer & Leader, 1994) was used to record en-route performance, 
option-selection and navigational patterns, time-on-lask data, time-in-program data, attitude data and posttest data. 

For the study, the content was divided into the five units, which taught nine objectives, as follows: 

A. Who Were the Anasazi? 

1. Identify the Anasazi, prehistoric Native Americans of the Southwest, from given descriptions. 

B. Where Did They Live? 

2. Locate where the Anasazi lived on a map. 

C. What Happened to the Anasazi? 

3. Identify the names of the two major phases of Anasazi development. 

4. Given a description of key events in Anasazi history, identify the name of the period of development. 

5. Identify what most agree happened to the Anasazi. 

D. How Did the Anasazi Live? 

6. Identify the type of villages the Anasazi lived in at the height of their culture. 

7. Identify features common to their homes, villages and lives. 

8. Identify kivas in pictures of villages, and identify their use. 

E. What Did They Use to Make a Living? 

9. Identify the uses of types of tools the Anasazi used around their homes, and for hunting. 

Each unit included instructional information and selected-response practice items. Information was presented in 
thf form of text, a reference map, a timeline, and graphic icons that could be clicked on to obtain facts about the icons. 
Sounds could also be accessed in conjunction with clicking on some of the graphics. 
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Students navigated through the program by using a mouse tool and selecting buttons, typically presented at the 
bottom of each screen. On the first screen of each unit, students could choose to review the objectives for that unit. 
Students could at any time choose to go the "Main Menu". The "Main Menu" button gave students free access to move 
anywhere in the program at any time. Students could move to different topics, the practice sets, objective screens, or the 
posttest by selecting these options. They could also skip portions of instruction or practice items by going back to the 
menu or directly to the practice sets or quizzes. 

Each screen also contained arrow buttons in which the students could return to the card they just viewed, the 
previous card in the stack, or continue to another card in the program. Students could quit the program at any point and 
later to resume instruction where they finished by clicking the "Quit" button. 

In additioi> to the instructional content, the CAI program included introductory instructions for using the 
program and pre- and post-instructional attitude questionnaires. The four item pre-instructional questionnaire was 
designed to measure students* prior interest and prior experience with computers and CAI as well as preference for 
learning by lecture, video or pictures, or computers. The twenty-one item post-instructional questionnaire measured 
students* attitudes toward the program content, ease of use, format, and clarity, as well as perceived effort and motivation 
toward the topic. Additional data were collected on student attitudes and use-patterns through a "notebook" feature built 
into the software. Every fifteen minutes, the program paused and requested students to write comments about how they 
were using the program. Patterns of navigational behaviors, option selection and time were tracked by a built-in 
computer program which recorded every keypress from the students and which was developed to track patterns thought to 
enhance learning, in particular branching back and forth among instruction and practice activities. 

Two versions of the program, differing only in their “incentive” directions in the introduction, were used for the 
study. Instructional presentation, practice, feedback, and the questionnaires and posttest were identical in both versions. 
Each version was worth a total of 25 possible course points. Students in the “task-incentive” treatment were informed 
that they received all 25 course points for simply completing the program, regardless of their posttest score. Students in 
the “performance-incentive” treatment were told that they would earn points based on their scores on the program 
posttest; they would earn 25 points if they scored at least 10% on the posttest. 

Procedures 

Prior to receiving a program disk, students were given a ninety-minute presentation about CAI and a 
demonstration about how to use a Macintosh computer. Hands-on experience was not included in the presentation, 
however the demonstration included large-screen projected computer screen images and verbal explanations. The 
instructional program itself was integrated into the course as a regular assignment designed to introduce the students to 
computer-assisted instruction. 

After the presentation, each student was given a disk at random representing one of the two versions of the 
program. Separate instruction sheets which gave directions for using any of the campus computers were included with 
the disks. Students were instructed that they would have two weeks to complete this assignment, and would be given 
one lab day during one of the class sessions in which to complete it. Several experimenters and the course instructor 
were available at various computer sites to answer any questions about the program. Program disks were collected two 
weeks later in the next regular class session. 

Criterion Measures 

The program included a 20- item multiple-choice posttest which measured achievement on the objectives of the 
instructional program. 

The four-item pre-instructional questionnaire measured students’ prior interest, prior computer and CAI 
experience, and attitudes toward computers. The twenty-one item post-instructional attitude questionnaire assessed 
students’ satisfaction with the material, perceived effort, desire for more information, and continuing motivation. (The 
items included in these two instruments are included in Tables 2 and 3.) 

Design and Data Analysis 

Analysis of 'variance was used to analyze achievement and attitudinal effects of the two incentive structures. 
Backward multiple regression analysis was performed on the patterns of navigational behaviors and time as related to 
achievement. Chi-square analysis was used to determine the relationships among prior interest, experience, learning 
achievement and attitudes. Students’ perceptions of how they were using the program were categorized and frequencies 
tabulated. 
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Results 



Achievement 

Students’ posttest scores were relatively high, as shown in Table 1. The mean overall score was 17.26 out of , 
20, with the range of scores being from 1 1 to 20. Students in the performance incentive treatment (M = 17.36) did 
slightly better than those in the task incentive treatment (M = 17.16), but this difference was not significant. 



Table 1 

Posttest Scores bv Incentive Treatment 



Treatment 


Mean 


SD 


N 


Performance incentive 


17.36 


1.81 


55 


Task incentive 


17.16 


2.07 


55 


Total 


17.26 


1.94 


110 



Note: Posttest scores ranged from 1 1 to 20 of 20 possible. 



Attitudes 

Pre-instructional Attitudes . Students’ responses to the items on the pre-instructional questionnaire are shown in 
Table 2. As can be seen, students were relatively neutral (2.38 on a scale from 1 to 5) in their prior interest in the topic 
of early Native Americans in the Southwest. The two “experience” items used a scale of from 1 (a lot) to 3 (none/not at 
all). Students indicated they had little prior experience with the Macintosh computer (M = 2.15), and even less 
experience with CAI (M = 2.3 1 ). Students’ mean score of 1 .98 on the learning mode preference item indicated some 
preference for learning by computers over lecture or video. 



Table 2 

Pre-instructional Attitude Data 



Question 


Mean 


SD 


1. Interest in topic 


2.38 


.98 


2. Mac experience 


2.15 


.66 


3. Prior CAI experience 


2.31 


.66 


4. Learning mode preference 


1.98 


.73 



Note: N = 1 10, Q1 scores ranged from 1 (high) to 5 (low). Q2 and Q3 scores ranged from 1 (a lot) to 3 (none/not at 
all). (J4 scores were 1 (lecture), 2(computers), and 3 (video/media). 



Post-instructional Attitudes . Mean scores on the 21 items on the post-instructional attitude questionnaire are 
shown in Table 3. Scores ranged from 1 (Strongly Agree) to 5 (Strongly disagree). As can be seen, students’ attitudes 
were generally positive, with all mean scores being more positive than 3, or neutral. Students indicated somewhat 
stronger positive responses on items related to how easy the program was to use, such as “1 knew where to click” (M = 
1 .55) and “It was easy to more aiound in the program” (M = 1 .76). Students also indicated they felt the program was 
well-designed, for instance, “The objectives in this clearly describe the content and practice” (M = 1-74), “The practice 
helped me to learn the content’’ (M = 1 -65), and “The content in Anasazi was easy to understand” (M = 1 -58). Students 





indicated that they liked using the computer to leam this content, for example, ‘The computer program was a good way 
to leam about the Anasazi’* (M = 1 -64), “I like learning with the computer*’ (M = 1 -77), “If I had the opportunity, I 
would choose to leam about another topic using a computer program like this one” (M = 197), and “I would want to use 
another program like this one to leam about another subject” (M = 1.79). Not surprisingly, in that this was a computer 
course mainly for education majors, students were somewhat less positive, or more neutral, in their attitudes about the 
Anasazi content, for instance, “The content was interesting” (M = 2.04), “I am interested in learning more about the early 
Native Americans of the Southwest” (M = 2.54), and “If I had the opportunity, I would choose to leam more about the 
Anasazi” (M = 2.57). Finally students* mean attitude scores indicated different amounts of enjoyment and satisfaction 
with the five different units. The unit students enjoyed the most was “How Did the Anasazi Live” (M = 1-92) which 
included information about houses and villages and many sounds, followed by “What Did They Use to Make a Living?” 
which included many icons of tools which accessed information about each (M = 2.05). Next students enjoyed “What 
Happened to the Anasazi?” (M = 2.17), based on the timeline, and finally “Who Were the Anasazi” and “Where Did They 
Live?’ (M = 2.05). Students indicated an almost equally positive perception about how effectively each of the units was 
presented with mean scores ranging from 1.81 to 1.86. 

Students* post-instructional attitude mean scores overall were subjected to ANOVA to determine any incentive 
treatment effects on overall attitudes. Results on these overall mean scores were 40.60 for performance incentive and 
40.42 for task incentive; these differences were not significant. 

Results of correlational analyses with attitudedata . Chi-square correlational analysis among the pre- 
instructional attitude scores and posttest scores indicated only one significant positive relationship (p<.05), which was for 
prior experience with CAI. A similar relationship appeared in the results of the chi-square analysis among the post- 
instructional attitude scores and posttest scores. A significant positive relationship was also found between desire to 
leam other content using CAI and posttest scores. 



Table 3 

Post-instructional Attitude Data 



Question 


Mean 


SJ5 


1 . The objectives in this program clearly described the content & practice. 


1.74 


.60 


2. 1 used the objectives to guide me in the program. 


2.15 


.86 


3. The content in Anasazi was easy to understand. 


1.58 


.61 


4. The content was interesting. 


2.04 


.78 


5. The practice helped me to leam the content. 


1.65 


.74 


6. It was easy to move around in the program. 


1.76 


.83 


7. 1 knew where to click. 


1.55 


.72 


8. The computer program was a good way to leam about the Anasazi. 


1.64 


.73 


9. 1 would want to use another program like this one to leam 
about another subject. 


1.79 


.94 


10. 1 like learning with a computer. 


1.77 


.76 
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1 1. 1 am interested in learning more about the early native 
Americans of the Southwest. 


2.54 


.92 


12. If I had the opportunity, I would choose to learn more about the Anasazi. 


2.57 


.98 


13. If I had the opportunity, I would choose to learn about another topic 
using a computer program like this one. 


1.97 


.88 


14. 1 enjoyed learning the material in the “Who Were the Anasazi” 
and in the “Where Did They Live” sections. 


2.05 


.66 


15. The material in the “Who Were the Anasazi” and in the 
“Where Did They Live” sections was presented effectively. 


1.84 


.53 


16. 1 enjoyed learning the material in the “What Happened to the 
Anasazi” section. 


2.17 


.82 


17. The material in the “What Happened to the Anasazi” 
section was presented effectively. 


1.88 


.69 


18. 1 enjoyed learning the material in the “How Did the Anasazi Uve?” 
section. 


1.92 


.74 


19. The material in the “How Did the Anasazi Live?” section was 
presented effectively. 


1.81 


.66 


20. 1 enjoyed learning the material in the “What Did They Use to 
Make a Living?” section. 


2.05 


.75 


21. The material in the “What Did They Use to Make a Living?” 
section was presented effectively. 


1.86 


.66 



Note: N = 1 10. Scores ranged from 1 (Strongly Agree) to 5 (Strongly Disagree). 



Navigation and Choice Patterns 

Mean scores on time and learner navigation and choice patterns are shown in Table 4. Students chose to return 
to instruction from practice exercises an average of 2.8 times, and chose previous and back options an average of 12.8 
times. The average total amount of time spent by students on instruction and practice was about 28 minutes (1679 
seconds). The mean number of instruction and practice cards viewed was 193 per student. As the total program included 
133 cards, including instruction, direction, questionnaire and test screens, this number indicates that many students 
viewed instruction and practice cards more than once. 
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Tabic 4 

Mean Scores on Learner Time and Navigation Variables 



Variable Mean SD 

Number of Times Chose to Return 
to Instruction from Practice 

INSTRTRN 2.8 2.7 

Number of Times Chose 
Previous and Back 

PRRTCNT 12.8 19.6 



Time (in seconds) Spent on 
Instruction and Practice 

TOTTIME 1679 813 

Total Number of Instruction and 
Practice Cards Viewed 

TOTCOUNT 193 112 



Multiple regression analysis performed to investigate the predictive value of several student time and navigation 
patterns on posttest scores is shown in Table 5. This analysis revealed that both total time spent on instruction and 
practice cards (Tottime) and number of times previous or back seleciions were made (Pntent) were significant predictors 
of achievement (p<.05). 



Table 5 

Backward Regression Analysis on Choice and Navigation for Posttest Results 



Variable Remaining 


R Square 


F 


P 


pntent & tottime 


.06190 


3.530 


.0328 


pntent 


.04489 


5.07609 


.0263 



* Significant @ the .05 level 



Students* Perceptions of How They Used the Piogram 

Data gathered from the “notebook” feature is shown in Table 6. Ninety-eight of the 110 students used the 
notebook feature. The most ftrquently-citcd response described what they were doing in the program (86), such as 
answering questions or taking quizzes. Almost as many students (85) indicated what they liked about the program, 
while 33 described features they did not like about it. Many students (84) stated that they were learning about Anasazi 
Indians, while 33 students said that the program was difficult. Thirty students volunteered that they liked having the 
option to go back and review screens and information and 28 students stated that they were using different mouse tools. 
Twelve students wrote that they had no comment. 
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Table 6 

Students* Perceptions of How They Were Using the Anasazi Computer Program 



ResDonses 


Number of Resoonses 


Answering practice questions, taking practice 
quizzes, tal^g final quiz, reading text, doing 
surveys, taking notes. 


86 


Liked program, program is interesting, easy to use, 
information is clear, likes immediate feedback, likes 
pictures and sound. 


85 


Learning about Anasazi Indiaiis, going through 
program. 


84 


Program is hard and confusing, got lost in program, 
program is long and boring, don’t like program. 


33 


Would rather learn from person or book, problems 
using mouse, survey too long, notebook feature is 
bothersome. 


33 


Likes being able to go back and review information in 
program, using review options. 


30 


Using mouse, buttons, arrows, etc.. 


28 


No comment. 


12 



Note: N = 98 out of 1 10 subjects used the notebook feature. Subjects could mention a number of responses. Categories 

were collapsed. 



Discussion 

This study is one of the first to investigate learners’ patterns of choices and navigation pathways in a completely 
ffee-access network-structure hypermedia program. There were no effects in this study for incentive treatment on either 
performance or attitudes. This is likely to have been a result of the compressed high scores on the posttest. Most 
students did very well, with a mean of 17 out of 20. This brings to mind Clark’s (1983) contention that well-designed 
instructional methods ate the key to success. Both treatments included carefully-designed instruction, which matched the 
instructional objectives and test items to the lesson content. Students’ responses to attitude items also indicated their 
satisfaction with the instructional design of the CAI program. 

The results of this study also indicate that college students generally like CAI programs, even those which teach 
content they may initially have been neutral about. Contrary to expectations, it appears that their prior interest may not 
be related to their performance. Similarly, their attitudes toward the CAI program were generally favorable. These 
attitudes, however, were also not related to their perfonnance. 

Results of this study did indicate that there mzy be a relationship between performance and prior experience, not 
with computers, but specifically with routine integration of CAI into college courses to teach content. The results of 
this study indicate that the more experience students have with CAI the better this method may enhance their learning of 
content. 
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A similar relationship between pi'rformance scores and students* desire to learn other content using CAI may 
indicate that more able students like learning using computers better. This result, too, seems to support the idea of 
providing college students with more cxpcr>n;c using computers to learn, if that is to be the method of delivery in a 
course. 

Other results indicate that the total time spent on instruction and practice cards and the number of previous or 
back selections made were significant predictors of achievement. That time spent on instruction affected acldevement was 
not surprising. The effects of students’ decisions to go back to previous instruction, however may be an important 
finding for developers of CAI. Supporting data, in the form of students’ perceptions of how they used the program, also 
indicate that students liked having the option to go back to previous screens wherever and whenever they desired. 

Results of se/eral previous learner-control studies indicate that learners do not know how to manage their own 
learning to improve achievement (Pollack and Sullivan, 1990; Ross and Rakow, 1981; Hannafin & Sullivan, 1994). 
However, in these studies students were either required to make choices before the instruction began or as they were 
learning new material. Students were not given the option to manage their own review of previously viewed screens of 
information. The findings from the present study indicate that even though learners may not be able to make 
instructional choices that improve achievement prior to learning new material, collegiate learners may be able to judge in 
what fashion they need to review previously-viewed instruction and practice in order to achieve higher posttest scores. 

The results of one study cannot be taken as a prescription for designers or researchers. It is suggested that future 
studies be conducted investigating the effects of going back and other patterns of learners’ navigational behaviors in free- 
access CAI programs more deeply. Studies should also be conducted with other types of learners in other settings, to 
determine the learning and attitudinal effects of incentive treatments, and navigational patterns and choices, as well as the 
‘ relationships among prior interest, prior experience, attitudes and performance. 



Table? 

Post-instnictjonal Attitude Data by Treatment 



Treatment 


Mean 


SD 


N 


Performance incentive 


40.60 


10.01 


55 


Task incentive 


40.42 


8.91 


55 


Total 


40.51 


9.43 


110 
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